Information processing system, method and computer readable medium for determining whether moving bodies appearing in first and second videos are the same or not

ABSTRACT

[Problem] To provide an information processing system, an information processing method, and a program, whereby it is possible to suitably determine whether mobile bodies appearing in a plurality of photography devices are the same mobile body. 
     [Solution] Provided is an information processing system, comprising: an input unit ( 110 ) which receives an input of a video; a detection unit ( 120 ) which detects a mobile body which appears in a first video and a second video which a first photography device ( 200 ) and a second photography device ( 200 ) respectively photograph; a similarity computation unit ( 130 ) which derives a first probability that a mobile body which appears in the first video and a mobile body which appears in the second video are the same on the basis of similarity of feature values of the mobile bodies; a non-appearance probability computation unit ( 140 ) which derives a second probability that the mobile body which appears in the first video does not appear in the second video on the basis of time elapsed from the mobile body exiting the frame of the first video; and a person determination unit ( 150 ) for determining whether the mobile body which appears in the first video is the same as the mobile body which appears in the second video on the basis of a comparison between the first probability and the second probability.

TECHNICAL FIELD

The present invention relates, in some aspects, to an informationprocessing system, an information processing method, and a program.

BACKGROUND ART

Over recent years, systems that perform monitoring over a wide rangeusing videos captured by a plurality of video cameras (imagingapparatuses) have been devised. PTL 1, for example, discloses amonitoring system that tracks a moving body captured by each of aplurality of cameras. In the method described in PTL 1, using featurevalues such as brightness values, colors or the like of moving bodiesextracted from respective cameras, identicalness of the moving bodies isdetermined.

CITATION LIST Patent Literature

[PTL 1] Japanese Laid-open Patent Publication No. 2006-146378

SUMMARY OF INVENTION Technical Problem

However, in the method described in PTL 1, when another moving bodyhaving a feature value similar to that of a moving body to be trackedappears in a video of a camera before the moving body to be trackedappears in the video of the camera, the another moving body may heerroneously tracked as the same moving body as the moving body to betracked.

In view of the problem, some aspects of the present invention have beenachieved, and one object of the present invention is to provide aninformation processing system, an information processing method, and aprogram capable of suitably determining identicalness of moving bodiesappearing in a plurality of imaging apparatuses.

Solution to Problem

An information processing system according to the present inventionincludes input means for receiving inputs of videos captured by aplurality of imaging apparatuses, detection means for detecting a movingbody appearing in a first video captured by a first imaging apparatusamong the plurality of imaging apparatuses and a moving body appearingin a second video captured by a second imaging apparatus among theplurality of imaging apparatuses, first computation means for deriving,on the basis of similarity between a feature value of the moving bodyappearing in the first video and a feature value of the moving bodyappearing in the second video, a first probability indicating aprobability that both moving bodies are the same, second computationmeans for deriving a second probability indicating a probability thatthe moving body appearing in the first video does not appear in thesecond video on the basis of an elapsed time after the moving body exitsand disappears from the first video, and determination means fordetermining whether or not the moving body appearing in the first videoand the moving body appearing in the second video are the same on thebasis of a comparison between the first probability and the secondprobability.

An information processing method according to the present inventionexecuted by an information processing system includes a step ofreceiving inputs of videos captured by a plurality of imagingapparatuses, a step of detecting a moving body appearing in a firstvideo captured by a first imaging apparatus among the plurality ofimaging apparatuses and a moving body appearing in a second videocaptured by a second imaging apparatus among the plurality of imagingapparatuses, a step of deriving, on the basis of similarity between afeature value of the moving body appearing in the first video and afeature value of the moving body appearing in the second video, a firstprobability in which both moving bodies are the same, a step of derivinga second probability in which the moving body appearing in the firstvideo does not appear in the second video On the basis of an elapsedtime after the moving body exits and disappears from the first video,and a step of determining whether or not the moving body appearing inthe first video and the moving body appearing in the second video arethe same on the basis of a comparison between the first probability andthe second probability.

A program according to the present invention that causes a computer toexecute processing for receiving inputs of videos captured by aplurality of imaging apparatuses, processing for detecting a moving bodyappearing in a first video captured by a first imaging apparatus amongthe plurality of imaging apparatuses and a moving body appearing in asecond video captured by a second imaging apparatus among the pluralityof imaging apparatuses, processing for deriving, on the basis ofsimilarity between a feature value of the moving body appearing in thefirst video and a feature value of the moving body appearing in thesecond video, a first probability in which both moving bodies are thesame, processing for deriving a second probability in which the movingbody appearing in the first video does not appear in the second video onthe basis of an elapsed time after the moving body exits and disappearsfrom the first video, and processing for determining whether or not themoving body appearing in the first video and the moving body appearingin the second video are the same on the basis of a comparison betweenthe first probability and the second probability.

In the present invention, “unit”, “means”, “device”, or “system” doesnot refer simply to physical means but includes a case in which afunction possessed by the “unit”, “means”, “device”, or “system” isrealized with software. It is also possible that a function of one“unit”, “means”, “device”, or “system” is realized with two or morephysical means or devices, or alternatively functions of two or more“units”, “means”, “devices”, or “systems” are realized with one physicalmeans or device.

Advantageous Effects of Invention

According to the present invention, it is possible to provide aninformation processing system, an information processing method, and aprogram capable of suitably determining identicalness of moving bodiesappearing in a plurality of imaging apparatuses.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a diagram illustrating a specific example of a video of avideo camera.

FIG. 1B is a diagram illustrating a specific example of a video of avideo camera.

FIG. 2A is a chart illustrating a specific example of non-appearanceprobability.

FIG. 2B is a chart illustrating a specific example of non-appearanceprobability.

FIG. 3 is a functional block diagram illustrating a schematicconfiguration of a tracking system according to a first exemplaryembodiment.

FIG. 4 is a diagram illustrating a specific example of a display screen.

FIG. 5 is a flowchart illustrating a flow of processing executed by aninformation processing server illustrated in FIG. 1.

FIG. 6 is a block diagram illustrating a configuration of hardware wherethe information processing server illustrated in FIG. 1 is mountable.

FIG. 7 is a functional block diagram illustrating a schematicconfiguration of a monitoring device according to a second exemplaryembodiment.

DESCRIPTION OF EMBODIMENTS

Exemplary embodiments of the present invention will be described below.In the following description and the illustration of the drawingsreferred to, identical or similar reference signs are assigned toidentical or similar configurations, respectively.

1 First Exemplary Embodiment

FIG. 1 to FIG. 6 are figures for illustrating a first exemplaryembodiment. Referring to these figures, the present exemplary embodimentwill be described along the following flow. Initially, in “1.1”, anoutline of a determination method of a moving body according to thepresent system is described. Then, in “1.2”, an outline of a functionalconfiguration of a system, in “1.3”, a flow of processing, and in “1.4”,a specific example of a hardware configuration where the present systemis mountable are illustrated. Lastly, in “1.5” and thereafter,advantageous effects according to the present invention and others aredescribed.

1.1 Outline

The present exemplary embodiment relates to a system that tracks amotion of a moving body, for example, by analyzing videos captured by aplurality of imaging apparatuses (e.g. monitoring cameras) havingimaging ranges different from each other. Therefore, in the systemaccording to the present exemplary embodiment, when a moving bodydeterminable to be the same as a moving body having exited anddisappeared from a first imaging range appears in a second imagingrange, both moving bodies are associated as the same moving body. Inother words, the present system determines identicalness of movingbodies.

As specific examples of the moving body, a person, a car, a motorcycle,a bicycle, and the like are conceivable. However, in the followingdescription, a case in which a person is tracked is mainly described.Referring to FIG. 1A and FIG. 1B, a determination method foridenticalness of persons appearing in videos is considered below.

To determine identicalness of persons, conceivable is a method in which,for example, from images of persons appearing in respective videos,feature values such as brightness values or colors are extracted andthen persons having similarity of the feature values higher than athreshold are determined as the same person.

However, as illustrated in FIG. 1A, upon tracking a person appearing ina video of a camera A at a time T, when at a time T+1, a person havinghigh similarity to a person to be tracked appears first in a video of acamera B although the person to be tracked has not yet appeared in thevideo (has not yet arrived at an imaging range) of the camera B, theperson may be erroneously determined as the person to be tracked.

To prevent such an erroneous association, a method in which a thresholdfor a same person determination based on similarity is set to be high isalso conceivable. However, when the threshold is set to be high, it ishighly possible that a determination for the same person is notperformed even when a person to be tracked appears.

As illustrated in FIG. 1B, for example, a case where the camera A andthe camera B are close to each other and a probability in which a personother than a person to be tracked appears in the camera B is low isassumed. In this case, when a threshold for a similarity determinationis excessively high even though a highly accurate feature value isunable to be acquired, for example, due to the fact that a personappears small or at a different angle, a person appearing in the cameraB is not determined to be the same as a person to he tracked.

Therefore, in the system according to the present exemplary embodiment,person tracking (association of persons) between cameras is performed,considering not only similarity but also a probability in which atracking target does not appear. Detailed description is made below.

It is assumed that when a person appears in a video of the camera Bafter t second elapsed after a person to be tracked exits and disappearsfrom a video of the camera A, a probability in which the person to betracked and the person appearing in the video of the camera B are thesame person, the probability being computed on the basis of imageinformation, is designated as P. P is computed, for example, on thebasis of a degree of similarity between a feature value of a trackingtarget person image (hereinafter, also referred to simply as “a featurevalue of a person”) in a video of the camera A and a person of a videoappearing in the camera B, and the like. As the feature value, varioustypes of values such as a color or brightness, a shape, a posture, and ahistogram thereof are conceivable.

Further, it is assumed that a probability in which a person to betracked is considered to have not yet appeared in a video of the cameraB is designated as Q. Q is a function employing an elapsed time t afterexiting and disappearing from a video of the camera A as at least a partof variables, and Q is a monotonously non-increasing function withrespect to t. Further, a decreased width of Q(t) differs, depending on adistance between an imaging range of the camera A and an imaging rangeof the camera B.

FIG. 2A and FIG. 2B each illustrate a specific example of theprobability Q(t). FIG. 2A illustrates changes of Q(t) with respect tothe time t in which a distance from an imaging range of the camera A toan imaging range of the camera B is long, and FIG. 2B illustrateschanges of Q(t) with respect to the time t in which the distance fromthe imaging range of the camera A to the imaging range of the camera Bis shorter than that of FIG. 2A.

Referring to FIG. 2A and FIG. 2B, in the case of the time t=0, Q(t) is 1in both figures. The reason is that when the imaging range of the cameraA and the imaging range of the camera B are not overlapped, a person tobe tracked is not considered to appear in a video of the camera B at amoment of exiting and disappearing from the camera A.

Thereafter, with an increase of t, Q(t) decreases. However, the case ofFIG. 2B is larger than the case of FIG. 2A in the decreasing rate ofQ(t). The reason is that the distance from the imaging range of thecamera A to the imaging range of the camera B is shorter in the case ofFIG. 2B than in the case of FIG. 2A, and therefore it is conceivablethat it takes a shorter time, in the case of FIG. 2B, for a person to hetracked for a long time to appear in a video of the camera B afterexiting and disappearing from a video of the camera A.

In the system according to the present exemplary embodiment, on thebasis of such P and Q(t), when P>Q(t) is satisfied, a person appearingin the camera B is determined as a person to be tracked. As describedabove, when imaging ranges of the camera A and the camera B are close toeach other, Q(t) decreases (a decreased width with respect to the time tincreases), and therefore, even when P is small, a person appearing inthe camera B and a person appearing in the camera A are easilydetermined as the same person.

On the other hand, when the imaging ranges of the camera A and thecamera B are distant from each other, Q(t) increases (a decreased widthwith respect to the time t decreases), and therefore unless P issufficiently large, the person appearing in the camera B and the personappearing in the camera A are not determined as the same person. In thiscase, a value of P is updated by observing the person who continuouslyappears in the camera B. It is conceivable that when, for example, animage having high resolution or an image having imaging conditions suchas a posture, a direction, and the like close to those of an imagecaptured by the camera A is obtained, the value of P is updated bycomputing again a feature value in which the image is treated as apriority. On the other hand, with each elapsed time, a probability inwhich a person to be tracked does not appear in a video of the camera Bdecreases, and therefore a value of Q(t) also decreases. Therefore, whenanother appropriate person candidate is not present, P>Q(t) is satisfiedwith time and then a person appearing in the camera B is determined as aperson to be tracked.

In the above description, as information that changes a value of Q(t), adistance between imaging ranges of cameras has been cited, but inaddition thereto, a plurality of pieces of information that change thevalue of Q(t) are conceivable. It is conceivable to make aconsideration, for example, whether an appearance is made in anothercamera. When a person does not appear, even with an elapsed time, in avideo of another camera reachable from an imaging range of the camera A,a probability in which a person appearing in a video of the camera B isa person to be tracked is high, and therefore a value of Q(t) is allowedto decrease a decreased width with respect to the time t is increased).

When there is a gateway through which a person to be tracked is able tobe directed toward another location between the camera A and the cameraB, it is conceivable that a probability in which a person appears in animaging range of a camera B is low. Therefore, in this case, a value ofQ(t) is allowed to increase (a decreased width with respect to the timet is decreased).

Alternatively, when one or more persons appear in a video of the cameraA other than a person to be tracked and the one or more other personsappear in a video of the camera B, it is conceivable that a probabilityin which the person to be tracked also appears in the video of thecamera B is high, and therefore a value of Q(t) is allowed to decrease(a decreased width with respect to the time t is increased).

1.2 System Outline 1.2.1 Outline of the Entire System

With reference to FIG. 3, a system configuration of a tracking system 1equivalent to the information processing system according to the presentexemplary embodiment will he described below. FIG. 1 is a block diagramillustrating a system configuration of the tracking system 1.

The tracking system 1 roughly includes an information processing server100, a plurality of video cameras 200A to 200N that each capture (image)a video (moving image), a display device 300, and an input device 400.The video cameras 200A to 200N are collectively referred to also as thevideo camera 200.

The tracking system 1 will be described below as a system for tracking aperson captured by the video camera 200. However, as described above, itis also conceivable that a tracking target is allowed to be varioustypes of moving bodies such as a car, a bicycle, a motorcycle, ananimal, and the like.

The video camera 200 equivalent to an imaging apparatus captures a videoand also determines whether or not there is a person in the capturedvideo. In addition thereto, the video camera 200 transmits informationsuch as a location and a feature value related to the person to theinformation processing server 100, together with the captured video.Through a comparison of the captured video between frames, the videocamera 200 can perform person tracking in the video.

It is possible that processing such as detection of a person (detectionof an image area related to the person) and extraction of a featurevalue from a video, person tracking in a camera, and the like isexecuted on, for example, the information processing server 100 oranother information processing device, not illustrated, other than thevideo camera 200.

The information processing server 100 analyzes the video captured by thevideo camera 200 and thereby executes various types of processing suchas detection of a person, registration of a person to be tracked,tracking of a registered person, and the like.

A case in which person tracking is performed on the basis of a real-timevideo captured by the video camera 200 is mainly described below, butwithout limitation thereto, it is also conceivable to perform tracking(analysis), for example, for a video captured by the video camera 200and then stored on a storage device (e.g., an HDD (Hard Disk Drive), VCR(Video Cassette Recorder), or the like). Further, it is also conceivableto perform tracking by reproducing in a reverse order (by reverselyreproducing) the video stored on the storage device. In general, when acertain person has exhibited suspicious behavior, it is necessary toexamine via what route the person has moved until the behavior and whatbehavior the person has exhibited, and therefore it is very useful toenable tracking by such reverse reproduction to be performed.

In person tracking using the information processing server 100, theinformation processing server 100 outputs a monitor screen to thedisplay device 300 and also inputs operation signals for various typesof operation inputs related to the person tracking from the input device400. More specifically, on the monitor screen displayed in the displaydevice 300, for example, a person to be tracked in a video input fromthe video camera 200 is displayed as a graphical user interface (GUI). Aspecific example of the monitor screen displayed by the display device300 will be described later with reference to FIG. 4.

To provide such a monitor screen to the user, the information processingserver 100 functions to determine whether or not a person appearing in avideo is a person to be tracked (whether or not the person appearing inthe video and the person to be tracked are the same person).

The display device 300 is a display that displays an image on, forexample, a liquid crystal, an organic EL (Electro Luminescence), or thelike. The monitor screen output from the information processing server100 is displayed by the display device 300 as describe above.

The input device 400 is a device for inputting various pieces ofinformation by the user. For example, a pointing device, including amouse, a touch pad, or a touch panel, a keyboard, and the likecorrespond to the input device 400.

Various types of configurations are conceivable for the informationprocessing server 100, the display device 300, and the input device 400.It is also conceivable that the display device 300 and the input device400 are realized as one client device. Alternatively, it is possible torealize functions possessed by the information processing server 100,the display device 300, and the input device 400 using four or moreinformation processing devices. When the display device 300 and theinput device 400 are realized as one client device, a part of functionsof the information processing device 100 according to the presentexemplary embodiment may be included in the client device.

1.2.2 Functional Configuration of Information Processing Server 100

A configuration of the information processing server 100 according tothe present exemplary embodiment will be described below. Theinformation processing server 100 includes, as illustrated in FIG. 3, aninput unit 110, a detection unit 120, a similarity computation unit 130,a non-appearance probability computation unit 140, a persondetermination unit 150, a display control unit 160, and a database (DB)170. A function of the information processing server 100 may he realizedusing a plurality of information processing devices (computers).

The input unit 110 outputs a video received from the video camera. 200to the display control unit 160 to display the video on the displaydevice 300. Further, the input unit 110 registers information of aperson detection result received in the same manner from the videocamera 200 on the DB 170 as detected person information 173. Thedetected person information 173 registered on the DB 170 by the inputunit 110 includes information of a feature value of a person detected bythe video camera 200.

The detection unit 120 detects, on the basis of the detected personinformation 173 indicating the person detection result for the videoinput from the video camera 200, whether or not a person appears in thevideo. When a person appears in the input video, the detection unit 120transmits the fact to the similarity computation unit 130 and thenon-appearance probability computation unit 140.

The similarity computation unit 1.30 computes similarity between afeature value of a person to be processed (e.g., a person appearing in avideo displayed on the display device 300) and a feature value of aperson to be tracked (a person caught by the video camera 200 beforethat time), using the feature value registered as the detected personinformation 173. When the similarity is high, a probability in whichboth persons are the same person is high. Therefore, the similaritycomputation unit 130 can use the similarity as a probability P in whichthe both persons are the same person, as described in the above “1.1”.

A feature value related to a person to be processed (a person appearingin the video camera 200 to be processed) is updated, as needed, with amovement of a person (a change in the size of the person on the video, achange in the posture and direction of the person, or the like).Therefore, the similarity computation unit 130 computes similarity, asneeded, on the basis of an updated feature value.

The non-appearance probability computation unit 140 computes aprobability (non-appearance probability) in which a person to be trackedhas not yet appeared in an imaging range to be processed. Theprobability computed by the non-appearance probability computation unit140 decreases with an increase in an elapsed time t from a time when theperson to be tracked has exited and disappeared from a video of thevideo camera 200 and is equivalent to Q(t) described in the above “1.1”.Q(t) can be changed according to information such as a distance betweenan imaging range of the video camera 200 where a person to be trackedhas finally exited and disappeared and an imaging range of the videocamera 200 to be processed, whether or not an appearance is made inanother video camera 200, and whether or not a person located in avicinity of the person to be tracked has already appeared in a video ofthe video camera 200 to be processed.

The person determination unit 150 compares a probability P equivalent tothe similarity computed by the similarity computation unit 130 and anon-appearance probability Q(t) computed by the non-appearanceprobability computation unit 140. The person determination unit 150determines, on the basis of the comparison result, whether or not theperson appearing in the video camera 200 to be processed is the same asthe person to he tracked. More specifically, as described above, whenP>Q(t) is satisfied, the person determination unit 150 can determine theperson appearing in the video camera 200 to be processed as the personto be tracked. On the other hand, when P≤Q(t) is satisfied, the persondetermination unit 150 can determine that a probability in which theperson appearing in the video camera 200 to be processed is the same asthe person to he tracked is low.

The display control unit 160 generates a monitor screen including avideo of the video camera 200 input by the input unit 110, a GUIindicating a same person determination result (i.e., whether to be aperson to be tracked) by the person determination unit 150, and others.The display control unit 160 outputs the generated monitor screen to thedisplay device 300.

A specific example of the monitor screen generated by the displaycontrol unit 160 is described with reference to FIG. 4. Since aprobability P is larger than a predetermined threshold, a probability inwhich a person is determined as the same person is high, but when it isdifficult to perform a determination for the same person since theprobability P is smaller than Q(t), it is possible that a state(tentative association state) of the person is indicated using arectangle of a dotted line as illustrated on the left side of FIG. 4 andwhen P has become larger than Q(t), the rectangle surrounding the personis surrounded with a solid line as illustrated on the right side of FIG.4 to notify the user of the fact that the person appearing in a videohas been determined as a person to be tracked. In this case, whenanother person more likely to be the person to he tracked appears in thevideo of the video camera 200, it is conceivable that the person to betentatively associated (the person indicated to the user using arectangle of a dotted line) is shifted.

The DB 170 manages probability information 171 and detected personinformation 173. The probability information 171 is information such asa function to compute a non-appearance probability Q(t) for whichspecific examples have been illustrated in FIG. 2A and FIG. 2B. Thenon-appearance probability computation unit 140 provides informationsuch as an elapsed time t and the like to the probability information171 to compute a value of the non-appearance probability Q(t).

The detected person information 173 is information on a person capturedand detected by the video camera 200. The detected person information173 includes information about via which route a movement has been made,a feature value for a detected person, and the like.

1.3 Flow of Processing

Next, a flow of processing of the information processing server 100 willbe described with reference to FIG. 5. FIG. 5 is a flowchartillustrating a flow of processing of the information processing server100 according to the, present exemplary embodiment.

Respective processing steps to be described below are executable bychanging orders optionally or in parallel unless a contradiction in theprocessing contents occur, and another step may be added between therespective processing steps. Further, a step described as a single stepfor convenience is executable by being divided into a plurality ofsteps, and steps described as a plurality of divided steps forconvenience are executable as a single step.

Initially, the detection unit 120 detects, on the basis of informationfrom the video camera 200 received in the input unit 110, that a personto be tracked has exited and disappeared from a video captured by thevideo camera 200 (hereinafter, this video camera :200 will be referredto also as the “camera A”) (S501). When the person to be tracked hasexited and disappeared, the non-appearance probability computation unit140 measures an elapsed time thereafter.

The detection unit 120 determines whether or not a person has appearedin a video of the video camera 200 (hereinafter, this video camera 200will be referred to also as the “camera B”) reachable by the person froman imaging area of the camera A (S503). When the person to be trackedhas appeared in the camera B (Yes in S503), the similarity computationunit 130 reads a feature value on the basis of an image of the person tobe tracked captured by the camera A or before that time and a featurevalue of the target person having appeared in the camera B from thedetected person information 173 of the DB 170 and computes similarity ofboth persons. The similarity computation unit 130 determines aprobability P in which the both persons are the same person from thesimilarity (S505).

On the other hand, the non-appearance probability computation unit 140computes an elapsed time t after the person to be tracked has exited anddisappeared from the image of the camera A and computes, on the basis ofthe elapsed time and the probability information 171 stored on the DB170, a probability Q(t) in which the person to be tracked appears in avideo of the camera B at the elapsed time t (S507).

The person determination unit 150 compares the probability P computed inthe similarity computation unit 130 and Q(t) computed in thenon-appearance probability computation unit 140 (S509). As a result,when P is larger than Q(t) (Yes in S509), the person appearing in thecamera B can be determined to be the same as the person to be tracked.Therefore, the person determination unit 150 associates both persons asthe same person and then continues tracking (S511).

On the other hand, when P is equal to or smaller than Q(t) (No in S509),the person determination unit 150 determines whether or not anotherperson (a person other than the person to be processed) appearing in thecamera B or each person appearing in a video being captured by anothervideo camera 200 at the current time has been associated with the personto be tracked (has been determined to be the same person) (S513). Whenanother person has been associated with the person to be tracked (Yes inS513), the person to be processed appearing in the camera B is not theperson to be tacked and then the processing is terminated. When anyperson of respective persons appearing in each video at the current timehas not yet been associated with the person to be tracked (No in S513),the processing is returned to S505 and then repeated again on the basisof a newly extracted feature value and an elapsed time.

1.4 Hardware Configuration

One example of a hardware configuration in which the informationprocessing server 100 described above is realized using a computer willbe described with reference to FIG. 6. As described above, a function ofthe information processing server 100 can be also realized using aplurality of information processing devices.

As illustrated in FIG. 6, the information processing server 100 includesa processor 601, a memory 603, a storage device 605, an input interface(I/F) 607, a data I/F 609, a communication I/F 611, and a display device613.

The processor 601 executes a program stored on the memory 603 to controlvarious types of processing in the information processing server 100.For example, processing for the input unit 110, the detection unit 120,the similarity computation unit 130, the non-appearance probabilitycomputation unit 140, the person determination unit 150, and the displaycontrol unit 160 described in FIG. 3 can be realized as a program mainlyoperating on the processor 601 by being temporarily stored on the memory603.

The memory 603 is, for example, a storage medium such as a RAM (RandomAccess Memory) or the like. The memory 603 temporarily stores programcodes of a program executed by the processor 601 and data necessary uponexecuting the program. In a storage area of the memory 603, for example,a stack area necessary during program execution is secured.

The storage device 605 is a non-volatile storage medium such as a harddisk, a flash memory, or the like. The storage device 605 stores anoperating system, various types of programs for realizing the input unit110, the detection unit 120, the similarity computation unit 130, thenon-appearance probability computation unit 140, the persondetermination unit 150, and the display control unit 160, and variouspieces of data including probability information 171 and detected personinformation 173 stored as the DB 170. The programs and the pieces ofdata stored on the storage device 605 are loaded on the memory 603 asneeded and referred to from the processor 601.

The input I/F 607 is a device for receiving an input from the user. Theinput device 400 described in FIG. 3 can be also realized using theinput I/F 607. Specific examples of the input I/F 607 include akeyboard, a mouse, a touch panel, and the like. The input I/F 607 may beconnected with the information processing server 100 via an interfacesuch as a USB (Universal Serial Bus) or the like.

The data I/F 609 is a device for inputting data from the outside of theinformation processing server 100. Specific examples of the data I/F 609include a drive device and the like for reading pieces of data stored onvarious types of storage media. It is also conceivable that the data I/F609 is disposed outside the information processing server 100. In thiscase, the data I/F 609 is connected with the information processingserver 100 via an interface such as a USB or the like.

The communication I/F 611 is a device for performing wired or wirelessdata communications to a device outside the information processingserver 100, such as the video camera 200 or the like. It is alsoconceivable that the communication I/F 611 is disposed outside theinformation processing server 1.00. In this case, the communication I/F611 is connected with the information processing server 100 via aninterface such as a USB or the like.

The display device 613 is a device for displaying various pieces ofinformation. The display device 300 described in FIG. 1 can be alsorealized using the display device 613. Specific examples of the displaydevice 613 include, for example, a liquid crystal display, an organic ELdisplay, and the like. The display device 613 may be disposed outsidethe information processing server 100. in this case, the display device613 is connected with the information processing server 100 via, forexample, a. display cable, or the like.

1.5 Advantageous Effects according to the Present Exemplary Embodiment

As described above, in the tracking system 1 according to the presentexemplary embodiment, not only similarity of a feature value of a personimage appearing in a video of the video camera 200 but also anon-appearance probability on the basis of an elapsed time after exitingand disappearing from a previous video are taken into account, andidenticalness of the person is determined (the person is tracked).Thereby, even if a feature value having sufficient accuracy is notacquired when a person appears in a video of the camera B after a personto be tracked has exited and disappeared from a video of the camera A,the person can be associated with the person to be tracked when imagingranges of the camera A and the camera B are close to each other and aprobability of an error is low. When a person to be tracked exits anddisappears from the camera A and immediately thereafter, a person havinga feature value similar to that of the person to be tracked appearsalthough a distance between the camera A and the camera B is long, theperson can be prevented from being erroneously associated with theperson to be tracked. In other words, it is possible to suitablydetermine identicalness of moving bodies appearing in a plurality ofimaging apparatuses.

2 Second Exemplary Embodiment

With reference to FIG. 7, a second exemplary embodiment will bedescribed below. FIG. 7 is a block diagram illustrating a functionalconfiguration of a monitoring device 700 that is an informationprocessing system. As illustrated in FIG. 7, the monitoring device 700includes an input unit 710, a detection unit 720, a first computationunit 730, a second computation unit 740, and a determination unit 750.The input unit 710 can receive an input of a video captured by animaging apparatus (e.g., a video camera) not illustrated.

The detection unit 720 detects a moving body appearing in a video(hereinafter, referred to as a first video) captured by a given videocamera (hereinafter, referred to as a first imaging apparatus) among aplurality of imaging apparatuses and a moving body appearing in a video(hereinafter, referred to as a second video) captured by another videocamera (hereinafter, referred to as a second imaging apparatus) amongthe plurality of imaging apparatuses.

The first computation unit 730 derives, on the basis of similaritybetween the moving body appearing in the first video and the moving bodyappearing in the second video, a probability (hereinafter, referred toas a first probability) in which both moving bodies are the same movingbody.

The second computation unit 740 derives, on the basis of an elapsed timeafter a moving body has exited and disappeared from the first video, aprobability (hereinafter, referred to as a second probability) in whichthe moving body (the moving body appearing in the first video) does notappear in the second video.

The determination unit 750 determines, on the basis of a comparisonbetween the first probability and the second probability, whether or notthe moving body appearing in the first video and the moving bodyappearing in the second video are the same.

When such mounting is performed, the monitoring device 700 according tothe present exemplary embodiment makes it possible to suitably determineidenticalness of moving bodies appearing in a plurality of imagingapparatuses.

3 Supplementary Matters

The configurations of the exemplary embodiments may be subjected tocombinations or replacements of a part of components. Further, theconstitution of the present invention is not limited to only theexemplary embodiments and can be subjected to various modificationswithout departing from the gist of the present invention.

A part or the whole of the exemplary embodiments can be also describedas in the following supplementary notes, but the present invention isnot limited to the following. Further, the program of the presentinvention may be a program that causes a computer to execute therespective operations described in the exemplary embodiments.

Supplementary Note 1

An information processing system including: input means for receivinginputs of videos captured by a plurality of imaging apparatuses;detection means for detecting a moving body appearing in a first videocaptured by a first imaging apparatus among the plurality of imagingapparatuses and a moving body appearing in a second video captured by asecond imaging apparatus among the plurality of imaging apparatuses;first computation means for deriving, on the basis of similarity betweena feature value of the moving body appearing in the first video and afeature value of the moving body appearing in the second video, a firstprobability in which both moving bodies are the same; second computationmeans for deriving a second probability in which the moving bodyappearing in the first video does not appear in the second video on thebasis of an elapsed time after the moving body exits and disappears fromthe first video; and determination means for determining whether or notthe moving body appearing in the first video and the moving bodyappearing in the second video are the same on the basis of a comparisonbetween the first probability and the second probability.

Supplementary Note 2

The information processing system according to Supplementary Note 1,wherein the determination unit determines that the moving body appearingin the first video and the moving body appearing in the second video arethe same when the first probability is larger than the secondprobability.

Supplementary Note 3

The information processing system according to Supplementary Note 1 orSupplementary Note 2, wherein the second probability is monotonouslynon-increasing with respect to an elapsed time after the moving bodyexits and disappears from the first video.

Supplementary Note 4

The information processing system according to Supplementary Note 3,wherein the second probability exhibits a less decreased width withrespect to an elapsed time when a distance from an imaging range of thefirst imaging apparatus to an imaging range of the second imagingapparatus is long than when the distance from the imaging range of thefirst imaging apparatus to the imaging range of the second imagingapparatus is short.

Supplementary Note 5

The information processing system according to any one of SupplementaryNote 1 to Supplementary Note 4, the system further including displaycontrol means for displaying the second video on a display device, thedisplay control means changing a display method, when the moving bodyappearing in the second video is the same as the moving body appearingin the first video, into a different display method from a displaymethod used before the moving bodies are determined to be the samemoving body.

Supplementary Note 6

The information processing system according to Supplementary Note 1 toSupplementary Note 5, wherein the determination means determines whetheror not any one of a plurality of moving bodies appearing in the firstvideo at the same time is the same as a part of moving bodies appearingin the second video, and the second computation means lowers the secondprobability for a moving body that is not associated with the movingbody appearing in the second video among the plurality of moving bodieswhen there is a moving body determined to be the same as the moving bodyappearing in the second video among the plurality of moving bodies.

Supplementary Note 7

An information processing method executed by an information processingsystem, the method including: a step of receiving inputs of videoscaptured by a plurality of imaging apparatuses; a step of detecting amoving body appearing in a first video captured by a first imagingapparatus among the plurality of imaging apparatuses and a moving bodyappearing in a second video captured by a second imaging apparatus amongthe plurality of imaging apparatuses; a step of deriving, on the basisof similarity between a feature value of the moving body appearing inthe first video and a feature value of the moving body appearing in thesecond video, a first probability in which both moving bodies are thesame; a step of deriving a second probability in which the moving bodyappearing in the first video does not appear in the second video on thebasis of an elapsed time after the moving body exits and disappears fromthe first video; and a step of determining whether or not the movingbody appearing in the first video and the moving body appearing in thesecond video are the same on the basis of a comparison between the firstprobability and the second probability.

Supplementary Note 8

The information processing method according to Supplementary Note 7, themethod determining that the moving body appearing in the first video andthe moving body appearing in the second video are the same when thefirst probability is larger than the second probability.

Supplementary Note 9

The information processing method according to Supplementary Note 7 orSupplementary Note 8, wherein the second probability is monotonouslynon-increasing with respect to an elapsed time after the moving bodyexits and disappears from the first video.

Supplementary Note 10

The information processing method according to Supplementary Note 9,wherein the second probability exhibits a less decreased width withrespect to an elapsed time when a distance from an imaging range of thefirst imaging apparatus to an imaging range of the second imagingapparatus is long than when the distance from the imaging range of thefirst imaging apparatus to the imaging range of the second imagingapparatus is short.

Supplementary Note 11

The information processing method according to any one of SupplementaryNote 1 to Supplementary Note 4, the method further including a step ofdisplaying the second video on a display device, the step changing adisplay method, when the moving body appearing in the second video isthe same as the moving body appearing in the first video, into adifferent display method from a display method used before the movingbodies are determined to be the same moving body.

Supplementary Note 12

The information processing method according to Supplementary Note 7 toSupplementary Note 11, the method determining whether or not any one ofa plurality of moving bodies appearing in the first video at the sametime is the same as a part of moving bodies appearing in the secondvideo, and lowering the second probability for a moving body that is notassociated with the moving body appearing in the second video among theplurality of moving bodies when there is a moving body determined to bethe same as the moving body appearing in the second video among theplurality of moving bodies.

Supplementary Note 13

A program that causes a computer to execute: processing for receivinginputs of videos captured by a plurality of imaging apparatuses;processing for detecting a moving body appearing in a first videocaptured by a first imaging apparatus among the plurality of imagingapparatuses and a moving body appearing in a second video captured by asecond imaging apparatus among the plurality of imaging apparatuses;processing for deriving, on the basis of similarity between a featurevalue of the moving body appearing in the first video and a featurevalue of the moving body appearing in the second video, a firstprobability in which both moving bodies are the same; processing forderiving a second probability in which the moving body appearing in thefirst video does not appear in the second video on the basis of anelapsed time after the moving body exits and disappears from the firstvideo; and processing for determining whether or not the moving bodyappearing in the first video and the moving body appearing in the secondvideo are the same on the basis of a comparison between the firstprobability and the second probability.

Supplementary Note 14

The program according to Supplementary Note 13, the program causing acomputer to determine that the moving body appearing in the first videoand the moving body appearing in the second video are the same when thefirst probability s larger than the second probability.

Supplementary Note 15

The program according to Supplementary Note 13 or Supplementary Note 14,wherein the second probability is monotonously non-increasing withrespect to an elapsed time after the moving body exits and disappearsfrom the first video.

Supplementary Note 16

The program according to Supplementary Note 15, wherein the secondprobability exhibits a less decreased width with respect to an elapsedtime when a distance from an imaging range of the first imagingapparatus to an imaging range of the second imaging apparatus is longthan when the distance from the imaging range of the first imagingapparatus to the imaging range of the second imaging apparatus is short.

Supplementary Note 17

The program according to any one of Supplementary Note 13 toSupplementary Note 16, the program further causing a computer to executeprocessing for displaying the second video on a display device and tochange changing a display method, when the moving body appearing in thesecond video is the same as the moving body appearing in the firstvideo, into a different display method from a display method used beforethe moving bodies are determined to be the same moving body.

Supplementary Note 18

The program according to Supplementary Note 13 to Supplementary Note 17,the program causing a computer to determine whether or not any one of aplurality of moving bodies appearing in the first video at the same timeis the same as a part of moving bodies appearing in the second video,and to lower the second probability for a moving body that is notassociated with the moving body appearing in the second video among theplurality of moving bodies when there is a moving body determined to bethe same as the moving body appearing in the second video among theplurality of moving bodies.

This application claims the benefit of priority based on Japanese patentapplication No. 2013-093849 filed on Apr. 26, 2013, the disclosure ofwhich is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   1 Tracking system-   100 Information processing server-   110 Input unit-   120 Detection unit-   130 Similarity computation unit-   140 Non-appearance probability computation unit-   150 Person determination unit-   160 Display control unit-   171 Probability information-   173 Detected person information-   200 Video camera-   300 Display device-   400 Input device-   601 Processor-   603 Memory-   605 Storage device-   607 Input interface-   609 Data interface-   611 Communication interface-   613 Display device-   700 Monitoring device-   710 Input unit-   720 Detection unit-   730 First computation unit-   740 Second computation unit-   750 Determination unit

1-8. (canceled)
 9. An information processing system comprising: at leastone memory storing instructions; and at least one processor configuredto execute the instructions to: receive inputs of videos captured by aplurality of imaging apparatuses; detect a first person image appearingin a first video captured by a first imaging apparatus among theplurality of imaging apparatuses and a second person image appearing ina second video captured by a second imaging apparatus among theplurality of imaging apparatuses; associate a first person with a secondperson, based on a first feature value of the first person and a secondfeature value of the second person; and provide an elapsed time from afirst time point when the first person disappears from the first videoto a second time point when the second person appears in the secondvideo, wherein the first feature value and the second feature valueinclude information based on histogram.
 10. The information processingsystem according to claim 9, wherein the elapsed time is provided afterthe first person associated with the second person, based on the firstfeature value of the first person and the second feature value of thesecond person.
 11. The information processing system according to claim9, wherein the at least one processor is further configured to executethe instructions to control a display device to display a image thatincludes the second person who is set as a monitoring target beinghighlighted, wherein a person appears in at least one of the first videoand the second video, and is associated with a person appearing inanother video.
 12. The information processing system according to claim11, wherein the at least one processor is further configured to executethe instructions to change a manner to highlight the second personincluded in the image, based on the elapsed time and a similaritybetween the first feature value and the second feature value.
 13. Theinformation processing system according to claim 12, wherein the atleast one processor is further configured to execute the instructionsto: compute the similarity, wherein the elapsed time is provided afterthe similarity computed.
 14. The information processing system accordingto claim 12, wherein the at least one processor is further configured toexecute the instructions to: compute a first probability indicating thatthe first person and the second person are same based on the similarity,and a second probability that the first person and the second person arenot the same based on the elapsed time, wherein the manner to highlightthe second person included in the image is changed, based on a result ofcomparison between the first probability and the second probability. 15.An information processing method performed by at least one computer, themethod comprising: receiving inputs of videos captured by a plurality ofimaging apparatuses; detecting a first person image appearing in a firstvideo captured by a first imaging apparatus among the plurality ofimaging apparatuses and a second person image appearing in a secondvideo captured by a second imaging apparatus among the plurality ofimaging apparatuses; associating a first person with a second person,based on a first feature value of the first person and a second featurevalue of the second person; and providing an elapsed time from a firsttime point when the first person disappears from the first video to asecond time point when the second person appears in the second video,wherein the first feature value and the second feature value includeinformation based on histogram.
 16. The information processing methodaccording to claim 15, wherein the elapsed time is provided after thefirst person associated with the second person, based on the firstfeature value of the first person and the second feature value of thesecond person.
 17. The information processing method according to claim15, further comprising: controlling a display device to display a imagethat includes the second person who is set as a monitoring target beinghighlighted, wherein a person appears in at least one of the first videoand the second video, and is associated with a person appearing inanother video.
 18. The information processing method according to claim17, further comprising: changing a manner to highlight the second personincluded in the image, based on the elapsed time and a similaritybetween the first feature value and the second feature value.
 19. Theinformation processing method according to claim 18, further comprising:computing the similarity, wherein the elapsed time is provided after thesimilarity computed.
 20. The information processing method according toclaim 18, further comprising computing a first probability indicatingthat the first person and the second person are same based on thesimilarity, and a second probability that the first person and thesecond person are not the same based on the elapsed time, wherein themanner to highlight the second person included in the image is changed,based on a result of comparison between the first probability and thesecond probability.
 21. A non-transitory computer readable recordingmedium storing programs, the programs causing at least one computer toperform: receiving inputs of videos captured by a plurality of imagingapparatuses; detecting a first person image appearing in a first videocaptured by a first imaging apparatus among the plurality of imagingapparatuses and a second person image appearing in a second videocaptured by a second imaging apparatus among the plurality of imagingapparatuses; associating a first person with a second person, based on afirst feature value of the first person and a second feature value ofthe second person; and providing an elapsed time from a first time pointwhen the first person disappears from the first video to a second timepoint when the second person appears in the second video, wherein thefirst feature value and the second feature value include informationbased on histogram.
 22. The non-transitory computer readable recordingmedium according to claim 21, wherein the elapsed time is provided afterthe first person associated with the second person, based on the firstfeature value of the first person and the second feature value of thesecond person.
 23. The non-transitory computer readable recording mediumaccording to claim 21, wherein the programs further causes the computerto perform: controlling a display device to display a image thatincludes the second person who is set as a monitoring target beinghighlighted, wherein a person appears in at least one of the first videoand the second video, and is associated with a person appearing inanother video.
 24. The non-transitory computer readable recording mediumaccording to claim 23, wherein the programs further causes the computerto perform: changing a manner to highlight the second person included inthe image, based on the elapsed time and a similarity between the firstfeature value and the second feature value.
 25. The non-transitorycomputer readable recording medium according to claim 24, wherein theprograms further causes the computer to perform: computing thesimilarity, wherein the elapsed time is provided after the similaritycomputed.
 26. The non-transitory computer readable recording mediumaccording to claim 24, wherein the programs further causes the computerto perform: computing a first probability indicating that the firstperson and the second person are same based on the similarity, and asecond probability that the first person and the second person are notthe same based on the elapsed time, wherein the manner to highlight thesecond person included in the image is changed, based on a result ofcomparison between the first probability and the second probability.